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Art Unit: 2123 

DETAILED ACTION: Non-Final Action 
Introduction 

1 . Title is: HARDWARE SIMULATION SYSTEMS AND METHODS FOR VISION 
INSPECTION SYSTEMS 

2. First named Inventor is: WASSERMAN. 

3. Claims 45-72 of US Application 09/243,689 filed on 2/3/99, are presented for examination. 

4. This Action is in response to the personal interview of 2/20/04. 

5. Prior 35 USC 102 rejections are replaced by 35 USC 103 rejections, and this action is made 
non-final. 

6. The finality of the prior action is withdrawn. 

Index and Definitions 

7. Stevenson refers to "Modeling optical vision systems with innovative software" by Michael 
Stevenson et al., Vision Systems Design, January 1999, pages 29-35 (from EDS). 

8. Thomas refers to US Patent 5,137,450 (from PTO form 892). 

9. Robotics refers to "Robotics" (Understanding Computers series) by Time-Life Books, 1986, 
ISBN 0-8094-5969-6 (from PTO form 892). Note that some additional pages (19-25) are 
presented with this action. 

35 USC § 112-Second Paragraph-indefinite claims: "off-line" 

10. The following is a quotation of the second paragraph of 35 U.S.C. 112: The specification shall 
conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the 
applicant regards as his invention. 

11. Claim 45-72 are rejected under 35 U.S.C. 112, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

12. Applicant has repeatedly asserted that the term "off-line programming system" in the 
preamble and in the body of claim 45 should be given patentable weight beyond the explicit 
limitations in the claim. The Examiner hereby gives the term some weight. 

13. Note that the term "off-line" appears to be a complex combination of negative and positive 
limitations, said limitations deriving their meaning from the context of the term. For 
example: off-line power generator, off-line computer, and off-line welder (removed from an 
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assembly line), or shutting down an assembly line (going off-line). Said examples have 
totally different meanings associated with the term "off-line". In other words, the term "off 
has many possible meanings, and the term "line" has many possible meanings. Further, 
negative limitations must be clearly defined and explicitly stated in the claim. Any positive 
limitation must be similarly defined and stated in the claim. 

14. For example, an "off-line emergency generator" at a hospital may have the negative 
limitation of not providing power when external power is available, and the positive 
limitation of automatically providing power to critical systems (like heart pumps) when no 
external power is available. 

15. Additionally, please state which portion of the original disclosure provides support for said 
positive and/or negative limitations. If said limitations are not supported by the original 
disclosure, then said amendments will be objected to as new matter. 

16. For the purposes of the 35 103 art rejection below, claim limitations E1-E3 are interpreted as 
being "off-line", and the Stevenson prior art is also interpreted as being off-line. 

17. Additionally, the claim 45 term "control instruction" is indefinite. It is not clear how the 
results of the simulation are used to generate a control instruction. And it is not clear what 
type of "instructions" are intended. For example, perhaps a keypad is the "control instruction 
generating portion" of claim 45, and the "control instructions" are generated by typing them 
into the keypad. Further, it is not clear how limitation D "control instruction generating 
portion" is patentably distinct from limitation E3 "[operable to] generate at least one control 
instruction". Please support any asserted distinction with citations from the original 
disclosure. 

18. Claims 46-72 are rejected for the same reasons as claim 45. 

Claim Rejections - 35 USC §103 

19. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: A patent may not be obtained though the invention is not 
identically disclosed or described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which said subject 
matter pertains. Patentability shall not be negatived by the manner in which the invention was made. 

20. The factual inquiries set forth in Graham v. John Deere Co,, 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: Determining the scope and contents of the prior art. 
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Ascertaining the differences between the prior art and the claims at issue. Resolving the level of ordinary skill 
in the pertinent art. Considering objective evidence present in the application indicating obviousness or 
nonobviousness. 

21. Claims 45-72 are rejected under 35 U.S.C. 103(a) as being unpatentable. 

22. Claim 45 is rejected under 35 U.S.C. 103(a) as being unpatentable over Stevenson in view of 
Robotics. 

23. Claim 45 is an independent "system" claim, with 5 limitations labeled A-E by the Examiner 
for clarity. Note that A has three subparts, B has 2 subparts, and E has 3 subparts. Said 
subparts are numerically labeled by the Examiner for clarity. 

24. Al-user interface... display a synthetic image is disclosed by Stevenson at page 29 
"Quality-control optical vision systems can be conveniently simulated and analyzed using 
software technology. . . simulate and analyze almost any optical vision system with high 
geometric and photometric accuracy". Note the car models displayed (synthetic images) at 
page 29 and page 30. Further note that these displays show tool bars and legends which are 
part of the user interface. 

25. A2-[user interface. . .] at least one control element that affects the focus is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". 

26. A3-[user interface. . .] input portion for receiving CAD data is disclosed by Stevenson at 
page 30 "This allows objects characterized in the Rhinoceros 3-D modeling program to be 
imported into ASAP for further analysis." 

27. Bl -hardware component simulation system... first portion... including a limited depth 
of field of the sense system is disclosed by Stevenson at page 32 "functional limitations 
inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution". 

28. B2-[hardware component simulation system]... second portion... relative position is 
disclosed by Stevenson at page 32 "functional limitations inherent to their system. In this 
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situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

29. C-communication interface... between the user interface and the hardware component 
simulation system is inherently disclosed by Stevenson at page 32 "functional limitations 
inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution". Note that the displays at pages 29 and 30 inherently 
have a communication interface between the display monitor and the simulation system. 
Similarly, the keyboard and/or mouse which is used to interact with the toolbars inherently 
has a communication interface between the display monitor and the simulation system. 

30. El-is operable to. . . focus dependent synthetic image. . . based on at least 2 of a current 
state of the user-alterable control elements [focus], the current lens system 
representation of the first portion [depth of field], and the current state representation 
of the second portion [relative position] is disclosed by Stevenson at page 32 "This type of 
analysis also permits designers to dissect the functional limitations inherent to their system. 
In this situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

31. E2-display the current focus-dependent synthetic image is disclosed by Stevenson at page 
29 "Quality-control optical vision systems can be conveniently simulated and analyzed using 
software technology. . . simulate and analyze almost any optical vision system with high 
geometric and photometric accuracy". Note the car models displayed (synthetic images) at 
page 29 and page 30. 

32. E3-generate at least one control instruction is disclosed by Stevenson at page 32 "This 
type of analysis also permits designers to dissect the functional limitations inherent to their 
system. In this situation it is possible to decide the proper light-source intensity and position, 
imaging camera position, depth of focus, effective focal length, field of view, and CCD 
camera resolution". Note that to "decide" the "proper" parameters inherently means 
controlling them with instructions. 
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33. The additional limitations are not expressly disclosed by Stevenson. 

34. D-control instruction generating portion is disclosed by Robotics figures and text on page 
54 and 55, especially the top right figure on page 55 which states "using a remote control 
device called a teach pendant, a trainer maneuvers a robot to one of many desired positions". 
Note that Robotics was supplied to the Applicant with the prior office action. Also see 
Robotics pages 19-25 which discuss robotic vision: particularly pages 22-23 which show 
robotic vision (or machine vision) for quality control of parts on an assembly line. 

35. Additionally, note that this "control instruction generating portion" limitation is implicitly 
disclosed by Stevenson at page 32 "This type of analysis also permits designers to dissect the 
functional limitations inherent to their system. In this situation it is possible to decide the 
proper light-source intensity and position, imaging camera position, depth of focus, effective 
focal length, field of view, and CCD camera resolution". Note that to "decide" the "proper" 
parameters implicitly means controlling them with instructions. If the machine vision being 
modeled by Stevenson is controlled by computers (which is implicit in a modern automobile 
manufacturing facility using robots), then said control would consist of writing new control 
instructions to computer control the proper parameters as listed in great detail by Stevenson. 

36. At the time the invention was made, it would have been obvious to a person of ordinary skill 
in the art to use Robotics to modify Stevenson. One of ordinary skill in the art would have 
been motivated to do this to implement Stevenson's machine vision simulation analysis of 
optimized parameters ("decide the proper light source intensity. . .") in a modern computer 
controlled factory using robots, as displayed in Robotics pages 21-25 and 54-55. Although 
not quite inherent, it is implicit that Stevenson's complex focus dependent machine vision 
simulation results will be implemented in a physical machine vision system that is controlled 
with computers. 

37. Claims 46-72 are rejected under 35 U.S.C. 103(a) as being unpatentable over Stevenson in 
view of Robotics and Thomas. 

38. In claim 46, "includes at least one control instruction... determines that a focus 
dependent actual inspection image... " is disclosed by Stevenson at page 32 "This type of 
analysis also permits designers to dissect the functional limitations inherent to their system. 
In this situation it is possible to decide the proper light-source intensity and position, imaging 
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camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". Note that Webster defines "depth of focus" as "the range of distances of the 
image behind a camera lens or other image-forming device measured along the axis of the 
device throughout which the image has acceptable sharpness". Thus, any part of an object 
positioned in the "depth of focus" will have "acceptable sharpness". Additionally, note page 
34 states "every optical-vision system has unique values determining its formal range of best 
focus". 

39. In claim 47, "control element that affects the focus of the synthetic image appears and 
operates substantially similarly to a control element included in a user interface of the 
machine vision inspection system" is disclosed by Thomas at FIG 3. Note that the Thomas 
flight simulator substantially duplicates the user interface of the actual airplane, including the 
control panel and control elements. Note Column 5 line 39 states "simulated heads-up 
display to complete the simulation for a typical tactical fighter". Thus, it is well known in 
the art to simulate the user interface as closely as possible for training purposes, even down 
to the pilot's chair. 

40. In claim 48, "the at least one control element that affects the focus of the synthetic image 
comprises at least one of a) a focusing control element... b) a motion control element" is 

disclosed by Stevenson at page 32 "This type of analysis also permits designers to dissect the 
functional limitations inherent to their system. In this situation it is possible to decide the 
proper light-source intensity and position, imaging camera position, depth of focus, effective 
focal length, field of view, and CCD camera resolution". Note that "light-source. . . 
position" implies that the light-source may be moved. Note that "imaging cameral position" 
implies that the camera position may be moved. Further, note that it is inherent that adjusting 
focus requires physically moving the physical lenses. 

41. In claim 49, "plurality of lenses", is disclosed by Stevenson at page 32 "This type of 
analysis also permits designers to dissect the functional limitations inherent to their system. 
In this situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". Note that if the physical system contained a plurality of lenses, then the 
analysis would inherently model the plurality of lenses. Further note that the use of lenses in 
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series is well known in the art (telescopes), and the use of swapping lenses in a system is also 
well known in the art (microscopes). 

42. In claim 50, "displays a modified current focus dependent synthetic image in response to 
a modification of at least one of a) the current state of the user-alterable control 
elements..." is disclosed by Stevenson at page 32 "This type of analysis also permits 
designers to dissect the functional limitations inherent to their system. In this situation it is 
possible to decide the proper light-source intensity and position, imaging camera position, 
depth of focus, effective focal length, field of view, and CCD camera resolution". Note that 
Claim 45 limitation A2 defines "user-alterable control elements" as "comprising at least one 
control element that affects the focus". 

43. Claim 51 contains three additional limitations: 

44. A-graphical user interface including user-alterable control elements and an image 
display portion is disclosed by Stevenson at the figure on page 29. Note the toolbars that 
form part of the graphical user interface. 

45. B-the user interface of the off-line programming system is substantially similar to the 
user interface of the machine vision inspection system is disclosed by Thomas at FIG 3. 
Note that the Thomas flight simulator substantially duplicates the user interface of the actual 
airplane, including the control panel and control elements. Note Column 5 line 39 states 
"simulated heads-up display to complete the simulation for a typical tactical fighter". Thus, 
it is well known in the art to simulate the user interface as closely as possible for training 
purposes, even down to the pilot's chair. If the actual system being replicated contained a 
graphical user interface, then the simulation trainer would also have a similar graphical user 
interface. Additionally, note that Specification page 1 line 21 states "Off-line programming 
software tools are popular". The phrase "off-line" implies that the actual user interface is 
used, while only the measuring machines and robots are simulated. 

46. C-majority of the user-alterable control elements typically appear and operate 
substantially similarly in both the off-line programming system and the machine vision 
inspection system is disclosed by Thomas at FIG 3. Note that the Thomas flight simulator 
substantially duplicates the user interface of the actual airplane, including the control panel 
and control elements. Additionally, note that Specification page 1 line 21 states "Off-line 
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programming software tools are popular". The phrase "off-line" implies that the actual user 
interface is used, while only the measuring machines and robots are simulated. 

47. Claim 52 contains three additional limitations: 

48. A-a third portion operable to represent a current state of a lighting system of the 
machine vision inspection system is disclosed by Stevenson at page 32 "This type of 
analysis also permits designers to dissect the functional limitations inherent to their system. 
In this situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

49. B-the user alterable control elements comprise at least one control element that affects 
the apparent lighting in the synthetic image representative of an image acquired by the 
machine vision system is disclosed by Stevenson at page 32 "This type of analysis also 
permits designers to dissect the functional limitations inherent to their system. In this 
situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". Note that Stevenson's "decide the proper light-source intensity" inherently 
implies a control element that affects the apparently lighting. 

50. C-the off-line programming system is operable to generate the current focus-dependent 
synthetic image based on at least three of [1] a current state of the user alterable 
control elements, [2] the current lens system representative of the first portion 
[represent at least a current lens system... including a limited depth of field of the lens 
system], [3] the current state representation of the second portion [represent a current 
state of at least the relative position] and [4] the current representation of the third 
portion [represent a current lighting system] is disclosed by Stevenson at page 32 "This 
type of analysis also permits designers to dissect the functional limitations inherent to their 
system. In this situation it is possible to decide the proper light-source intensity and position, 
imaging camera position, depth of focus, effective focal length, field of view, and CCD 
camera resolution". Note Stevenson discloses all four of the sub-limitations (1 through 4). 
Further note that Stevenson's term "decide" means that Stevenson's listed factors are 
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alterable, and that a current representation is generated based upon the current state of each 
factor. 

51. In claim 53, "the at least one control element that affects the apparent lighting in the 
synthetic image comprises a user-alterable control element of the machine vision 
inspection system, such that the at least one control element that affects the apparent 
lighting in the synthetic image appears and operates substantially similarly in both the 
off-line programming system and the machine vision inspection system is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". Note that some of Stevenson's parameters apper 
to be user-alterable, such as light-source position, and imaging camera position. Also see 
Stevenson page 30 "These features enable designers to visualize all the key elements of their 
system's optical performance". 

52. In claim 54, there are four additional limitations. 

53. A-input operation instructions which are substantially similar to the... machine vision 
inspection system is disclosed by Thomas at FIG 3. Note that the Thomas flight simulator 
substantially duplicates the user interface of the actual airplane, including the control panel 
and control elements. The physical user interface is duplicated, and the input operation 
instructions are implicitly duplicated as well. Note that Thomas' goal is "realistic flight 
simulation" at Column 1 line 17. 

54. B-hardware component simulation system processes the input operation instructions in 
order to generate the current focus-dependent synthetic image is disclosed by Stevenson 
at page 32 "This type of analysis also permits designers to dissect the functional limitations 
inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution". 

55. C-the user-alterable control elements include elements operable to input image 
inspection operation instructions substantially similar to at least one control instruction 
usable in the inspection program is disclosed by Thomas at FIG 3. Note that the Thomas 
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flight simulator substantially duplicates the user interface of the actual airplane, including the 
control panel and control elements. The physical user interface is duplicated, and the input 
operation instructions are implicitly duplicated as well Note that Thomas' goal is "realistic 
flight simulation" at Column 1 line 17. 

56. D-hardware component simulation system generates the current focus-dependent 
synthetic image... substantially similar to... the machine vision inspection system is 
disclosed by Thomas at FIG 3. Note that the Thomas flight simulator substantially duplicates 
the user interface of the actual airplane, including the control panel and control elements. 
The physical user interface is duplicated, and the input operation instructions are implicitly 
duplicated as well. Note that Thomas' goal is "realistic flight simulation" at Column 1 line 
17. 

57. At the time the invention was made, it would have been obvious to a person of ordinary skill 
in the art to use Thomas to modify Stevenson. One of ordinary skill in the art would have 
been motivated to do this to improve the training efficiency of the simulation by simulating 
the entire system as closely possible. Additionally, simply copying the user interface 
software modules into the simulation system would save developmental time. 

58. In claim 55, there are with 6 limitations labeled A-F by the Examiner. Limitation A has 2 
subparts, C has 2 subparts, and F has 3 subparts. 

59. Al-user interface... display a synthetic image is disclosed by Stevenson at page 29 
"Quality-control optical vision systems can be conveniently simulated and analyzed using 
software technology. . . simulate and analyze almost any optical vision system with high 
geometric and photometric accuracy". Note the car models displayed (synthetic images) at 
page 29 and page 30. Further note that these displays show tool bars and legends which are 
part of the user interface. 

60. A2-[user interface. . .] at least one control element that affects the focus is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". 
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61. A3-[user interface...] at least one control element... image inspection operation is 

disclosed by Stevenson at page 30 "visual geometry inspection" and page 35 "searches the 
image for hot spots". 

62. B-input portion for receiving CAD data is disclosed by Stevenson at page 30 "This allows 
objects characterized in the Rhinoceros 3-D modeling program to be imported into ASAP for 
further analysis." 

63. CI -hardware component simulation system... first portion... including a limited depth 
of field of the lense system is disclosed by Stevenson at page 32 "functional limitations 
inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution". 

64. C2-[hardware component simulation system]... second portion... relative position is 
disclosed by Stevenson at page 32 "functional limitations inherent to their system. In this 
situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

65. D-communication interface... between the user interface and the hardware component 
simulation system is inherently disclosed by Stevenson at page 32 "functional limitations 
inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution". Note that the displays at pages 29 and 30 inherently 
have a communication interface between the display monitor and the simulation system. 
Similarly, the keyboard and/or mouse which is used to interact with the toolbars inherently 
has a communication interface between the display monitor and the simulation system. 

66. E-control instruction generating portion is disclosed by Stevenson at page 32 "This type 
of analysis also permits designers to dissect the functional limitations inherent to their 
system. In this situation it is possible to decide the proper light-source intensity and position, 
imaging camera position, depth of focus, effective focal length, field of view, and CCD 
camera resolution". Note that to "decide" the "proper" parameters inherently means 
controlling them with instructions. 
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67. Fl-is operable to. . . focus dependent synthetic image. . . based on at least 2 of a current state 
of the user-alterable control elements [focus], the current lens system representation of 
the first portion [depth of field], and the current state representation of the second 
portion [relative position] is disclosed by Stevenson at page 32 "This type of analysis also 
permits designers to dissect the functional limitations inherent to their system. In this 
situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

68. F2-display the current focus-dependent synthetic image is disclosed by Stevenson at page 
29 "Quality-control optical vision systems can be conveniently simulated and analyzed using 
software technology. . . simulate and analyze almost any optical vision system with high 
geometric and photometric accuracy". Note the car models displayed (synthetic images) at 
page 29 and page 30. 

69. F3-perform an image inspection operation based on the current focus-dependent 
synthetic image is disclosed by Stevenson at page 30 "visual geometry inspection" and page 
35 "searches the image for hot spots". 

70. In claim 56, "generate at least one control instruction usable in an inspection program" 
is disclosed by Robotics figures and text on page 54 and 55, especially the top right figure on 
page 55 which states "using a remote control device called a teach pendant, a trainer 
maneuvers a robot to one of many desired positions". Note that Robotics was supplied to the 
Applicant with the prior office action. Also see Stevenson at page 30 "visual geometry 
inspection" and page 35 "searches the image for hot spots", which implicitly would be 
implemented on the actual physical system using computer controls, if the machine vision 
system was in a modern manufacturing facility. 

71 . In claim 57, there are 5 limitations labeled A-E by the Examiner. Limitation A has 4 
subparts, C has 2 subparts, and E has 3 subparts. 

72. Al-"user interface substantially similar to the graphical user interface of the machine 
vision inspection system... display a synthetic image" is disclosed by Thomas at FIG 3. 
Note that the Thomas flight simulator substantially duplicates the user interface of the actual 
airplane, including the control panel and control elements. The physical user interface is 
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duplicated, and any graphical user interface is implicitly duplicated as well. Note that 
Thomas' goal is "realistic flight simulation" at Column 1 line 17. One of ordinary skill in the 
art would understand that realistic simulation requires duplicating all interfaces as exactly as 
possible, including graphical user interfaces. Note that graphical user interfaces are well 
known in the art, and are defined by Computer User Dictionary (1998) as "A type of 
environment that represents programs, files, and options by means of icons, menus, and 
dialog boxes on the screen. The user can select and activate these options by pointing and 
clicking with a mouse or, often, by using the keyboard. . .". 

73. A2-[user interface. . .] at least one control element that affects the focus is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". 

74. A3-[user interface...] at least one control element... image inspection operation is 
disclosed by Stevenson at page 30 "visual geometry inspection" and page 35 "searches the 
image for hot spots". 

75. A4-majority of the user-alterable control elements typically appear and operates 
substantially similarly in both the machine vision inspection simulation system and the 
machine vision... is disclosed by Thomas at FIG 3. Note that the Thomas flight simulator 
substantially duplicates the user interface of the actual airplane, including the control panel 
and control elements. The physical user interface is duplicated, and any graphical user 
interface is implicitly duplicated as well. Note that Thomas' goal is "realistic flight 
simulation" at Column 1 line 17. One of ordinary skill in the art would understand that 
realistic simulation requires duplicating all interfaces as exactly as possible, including the 
control elements. 

76. B-input portion for receiving CAD data is disclosed by Stevenson at page 30 "This allows 
objects characterized in the Rhinoceros 3-D modeling program to be imported into ASAP for 
further analysis." 

77. CI -hardware component simulation system... first portion... including a limited depth 
of field of the lens system is disclosed by Stevenson at page 32 "functional limitations 
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inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution". 

78. C2-[hardware component simulation system]... second portion... relative position is 
disclosed by Stevenson at page 32 "functional limitations inherent to their system. In this 
situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

79. D-communication interface... between the user interface and the hardware component 
simulation system is inherently disclosed by Stevenson at page 32 "functional limitations 
inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution". Note that the displays at pages 29 and 30 inherently 
have a communication interface between the display monitor and the simulation system. 
Similarly, the keyboard and/or mouse which is used to interact with the toolbars inherently 
has a communication interface between the display monitor and the simulation system. 

80. El-is operable to. . . focus dependent synthetic image. . . based on at least 2 of a current 
state of the user-alterable control elements [focus], the current lens system 
representation of the first portion [depth of field], and the current state representation 
of the second portion [relative position] is disclosed by Stevenson at page 32 "This type of 
analysis also permits designers to dissect the functional limitations inherent to their system. 
In this situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

81. E2-display the current focus-dependent synthetic image is disclosed by Stevenson at page 
29 "Quality-control optical vision systems can be conveniently simulated and analyzed using 
software technology. . . simulate and analyze almost any optical vision system with high 
geometric and photometric accuracy". Note the car models displayed (synthetic images) at 
page 29 and page 30. 
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82. E3-perform an image inspection operation based on the current focus-dependent 
synthetic image is disclosed by Stevenson at page 30 "visual geometry inspection" and page 
35 "searches the image for hot spots". 

83. In claim 58, "plurality of lenses" is disclosed by Stevenson at page 29 "Quality-control 
optical vision systems can be conveniently simulated". The mathematics of multiple lenses 
in series is well known in the art, and common in telescopes and cameras. Additionally, 
swapping multiple lenses is well known in devices such as microscopes. 

84. In claim 59, "at least one of a) the current state of the user-alterable control elements 
[focus], b) the current lens system representation of the first portion [depth of field], 
and c) the current state representation of the second portion [relative position] 99 is 
disclosed by Stevenson at page 32 "This type of analysis also permits designers to dissect the 
functional limitations inherent to their system. In this situation it is possible to decide the 
proper light-source intensity and position, imaging camera position, depth of focus, effective 
focal length, field of view, and CCD camera resolution". 

85. Claim 60 has three additional limitations: 

86. A-"current state of a lighting system 99 is disclosed by Stevenson at page 32 "This type of 
analysis also permits designers to dissect the functional limitations inherent to their system. 
In this situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

87. B-"at least one control element that affects the apparent 99 is disclosed by Stevenson at 
page 32 "This type of analysis also permits designers to dissect the functional limitations 
inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution". 

88. C-"based on at least three of a) the current state of the user-alterable control elements 
[focus], b) the current lens system representation of the first portion [depth of field], 
and c) the current state representation of the second portion [relative position] and the 
current state representation of the third portion [image inspection] 99 is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
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limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution" and page 30 "visual geometry inspection" and 
page 35 "searches the image for hot spots". Note that the Examiner is interpreting "third 
portion" as meaning "image inspection" in view of parent claim 57. 

89. In claim 61, "a plurality of lights of the machine vision inspection system" is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". Although Stevenson does not explicitly disclose 
a plurality of lights, the Examiner takes Official Notice that multiple lights are often used in 
photography, and are necessary to clearly illuminate complex three dimensional objects. One 
of ordinary skill in the art of optical vision systems would be familiar with the use of 
multiple lights. Also note MPEP 2144.04(VI)(B). In re Harza, 274 F.2d 669, 124 USPQ 
378, 380 (CCPA 1960) states "It is well settled that the mere duplication of parts has no 
patentable significance unless a new and unexpected result is produced". 

90. The Applicant is entitled to traverse the official notice according to MPEP § 2144.03. 
However, MPEP § 2144.03 further states "See also In re Boon, 439 F.2d 724, 169 USPQ 231 
(CCPA 1971) (a challenge to the taking of judicial notice must contain adequate information 
or argument to create on its face a reasonable doubt regarding the circumstances justifying 
the judicial notice)." Specifically, In re Boon, 169 USPQ 231, 234 states "as we held in 
Ahlert, an applicant must be given the opportunity to challenge either the correctness of the 
fact asserted or the notoriety or repute of the reference cited in support of the assertion. We 
did not mean to imply by this statement that a bald challenge, with nothing more, would be 
all that was needed". Further note that 37 CFR § 1.671(c)(3) states "Judicial notice means 
official notice". Thus, a traversal by the Applicant that is merely "a bald challenge, with 
nothing more" will be given very little weight. 

91. In claim 62, 3 additional limitations: 

92. A-user-alterable control elements [focus]... substantially similar to... the machine 
vision inspection system is disclosed by Stevenson at page 32 "This type of analysis also 
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permits designers to dissect the functional limitations inherent to their system. In this 
situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

93. B-generate the current focus-dependent synthetic image is disclosed by Stevenson at page 
32 "This type of analysis also permits designers to dissect the functional limitations inherent 
to their system. In this situation it is possible to decide the proper light-source intensity and 
position, imaging camera position, depth of focus, effective focal length, field of view, and 
CCD camera resolution", and the displayed images at page 29 and page 30. 

94. C-generates the current focus-dependent synthetic image in a form which is operable 
with the at least one control element which is operable to perform an image inspection 
operation based on the current focus-dependent synthetic image to provide an off-line 
environment for training a part program based on a focus-dependent synthetic image 
that is substantially similar to an environment that is provided by the machine vision 
inspection system for training a part program based on a focus-dependent actual image 
is disclosed by Stevenson at page 32 "This type of analysis also permits designers to dissect 
the functional limitations inherent to their system. In this situation it is possible to decide the 
proper light-source intensity and position, imaging camera position, depth of focus, effective 
focal length, field of view, and CCD camera resolution", and Stevenson page 30 "These 
features enable designers to visualize all the key elements of their system's optical 
performance", and Stevenson page 29 "fine-tune". 

95. In claim 63, "further comprising a control instruction generating portion, wherein the 
machine vision inspection simulation system is further operable to generate at least one 
control instruction usable in an inspection program for the at least one object 
inspectable by the machine vision inspection system, based at least partially on the 
current state of the user-alterable control elements" is disclosed by Stevenson at page 32 
"This type of analysis also permits designers to dissect the functional limitations inherent to 
their system. In this situation it is possible to decide the proper light-source intensity and 
position, imaging camera position, depth of focus, effective focal length, field of view, and 
CCD camera resolution". Note that Stevenson's term "decide the proper. . . " means that the 
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simulated system is used to determine the "proper" settings or control instructions for the 
actual system. 

96. In claim 64, "external view representing the overall configuration of the machine vision 
inspection system" is disclosed by Stevenson at page 32 "This type of analysis also permits 
designers to dissect the functional limitations inherent to their system. In this situation it is 
possible to decide the proper light-source intensity and position, imaging camera position, 
depth of focus, effective focal length, field of view, and CCD camera resolution". 

97. In claim 65, 4 limitations A-D. Limitation A has 2 subparts, B has 2 subparts, and D has 4 
subparts: 

98. Al-user interface... display a synthetic image representative of an image acquired by 
the machine vision system is disclosed by Thomas at FIG 3, and at Column 5 line 39 
"simulated heads-up display". Note that the simulated heads-up display includes machine 
vision images such as radar acquired artificial horizon and enemy fighters. Note that the 
Thomas flight simulator substantially duplicates the user interface of the actual airplane, 
including the control panel and control elements. The physical user interface is duplicated, 
and any graphical user interface is implicitly duplicated as well. Note that Thomas' goal is 
"realistic flight simulation" at Column 1 line 17. One of ordinary skill in the art would 
understand that realistic simulation requires duplicating all interfaces as exactly as possible, 
including graphical user interfaces. Note that graphical user interfaces are well known in the 
art, and are defined by Computer User Dictionary (1998) as "A type of environment that 
represents programs, files, and options by means of icons, menus, and dialog boxes on the 
screen. The user can select and activate these options by pointing and clicking with a mouse 
or, often, by using the keyboard. . .". 

99. A2-[user interface. . .] at least one control element that affects the focus is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". 

100. Bl -hardware component simulation system... first portion... including a limited 
depth of field of the lens system is disclosed by Stevenson at page 32 "functional 
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limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". 

101. B2-hardware component simulation system... first portion... including a limited 
depth of field of the lens system is disclosed by Stevenson at page 32 "functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". 

102. C-communication interface... between the user interface and the hardware 
component simulation system is inherently disclosed by Stevenson at page 32 "functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". Note that the displays at pages 29 and 30 
inherently have a communication interface between the display monitor and the simulation 
system. Similarly, the keyboard and/or mouse which is used to interact with the toolbars 
inherently has a communication interface between the display monitor and the simulation 
system. 

103. Dl-inputting CAD data is disclosed by Stevenson at page 30 "This allows objects 
characterized in the Rhinoceros 3-D modeling program to be imported into ASAP for further 
analysis." 

104. D2- focus dependent synthetic image... based on at least 2 of a current state of the 
user-alterable control elements [focus], the current lens system representation of the 
first portion [depth of field], and the current state representation of the second portion 
[relative position] is disclosed by Stevenson at page 32 "This type of analysis also permits 
designers to dissect the functional limitations inherent to their system. In this situation it is 
possible to decide the proper light-source intensity and position, imaging camera position, 
depth of focus, effective focal length, field of view, and CCD camera resolution". 

105. D3 -displaying is disclosed by Stevenson at page 29 figure and page 30 figures. 

106. Degenerating at least one control instruction... based at least partially on the 
current state of the user-alterable control elements [focus] is disclosed by Stevenson at 
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page 32 "This type of analysis also permits designers to dissect the functional limitations 
inherent to their system. In this situation it is possible to decide the proper light-source 
intensity and position, imaging camera position, depth of focus, effective focal length, field 
of view, and CCD camera resolution" and page 30 "visual geometry inspection" and page 35 
"searches the image for hot spots". 

107. In claim 66, "altering the at least one control element that affects the focus of the 
synthetic image" is disclosed by Stevenson at page 32 "This type of analysis also permits 
designers to dissect the functional limitations inherent to their system. In this situation it is 
possible to decide the proper light-source intensity and position, imaging camera position, 
depth of focus, effective focal length, field of view, and CCD camera resolution". 

108. In claim 67, "displaying a modified current focus-dependent synthetic image... at 
least one of a) the current state of the user-alterable control elements [focus], b) the 
current lens system representation of the first portion [depth of field], and c) the 
current state representation of the second portion [relative position]" is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution" and page 30 "visual geometry inspection" and 
page 35 "searches the image for hot spots". 

109. In claim 68, "at least one control element... image inspection operation" is disclosed 
by Stevenson at page 32 "This type of analysis also permits designers to dissect the 
functional limitations inherent to their system. In this situation it is possible to decide the 
proper light-source intensity and position, imaging camera position, depth of focus, effective 
focal length, field of view, and CCD camera resolution" and page 30 "visual geometry 
inspection" and page 35 "searches the image for hot spots". 

110. In claim 69, 7 limitations A-G. Limitation A has 4 subparts, B has 2 subparts: 

111. Al-user interface... display a synthetic image representative of an image acquired 
by the machine vision system is disclosed by Thomas at FIG 3, and at Column 5 line 39 
"simulated heads-up display". Note that the simulated heads-up display includes machine . 
vision images such as radar acquired artificial horizon and enemy fighters. Note that the 
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Thomas flight simulator substantially duplicates the user interface of the actual airplane, 
including the control panel and control elements. The physical user interface is duplicated, 
and any graphical user interface is implicitly duplicated as well. Note that Thomas' goal is 
"realistic flight simulation" at Column 1 line 17. One of ordinary skill in the art would 
understand that realistic simulation requires duplicating all interfaces as exactly as possible, 
including graphical user interfaces. Note that graphical user interfaces are well known in the 
art, and are defined by Computer User Dictionary (1998) as "A type of environment that 
represents programs, files, and options by means of icons, menus, and dialog boxes on the 
screen. The user can select and activate these options by pointing and clicking with a mouse 
or, often, by using the keyboard. . .". 

112. A2-[user interface. . .] at least one control element that affects the focus is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". 

113. A3-[user interface. . .] at least one control element... image inspection is disclosed by 
Stevenson at page 30 "visual geometry inspection" and page 35 "searches the image for hot 
spots". 

1 14. A4-majority of the user-alterable control elements [focus] and image processing 
tools typically appear and operate substantially similarly is disclosed by Thomas at FIG 
3. Note that Thomas' goal is "realistic flight simulation" at Column 1 line 17. One of 
ordinary skill in the art would understand that realistic simulation requires duplicating all 
interfaces as exactly as possible, including control elements and image processing tools. 

115. Bl -hardware component simulation system... first portion... depth of field is 
disclosed by Stevenson at page 32 "This type of analysis also permits designers to dissect the 
functional limitations inherent to their system. In this situation it is possible to decide the 
proper light-source intensity and position, imaging camera position, depth of focus, effective 
focal length, field of view, and CCD camera resolution". 

116. B2-second portion... relative position and the [third] portion... inspection is 
disclosed by Stevenson at page 32 "This type of analysis also permits designers to dissect the 
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functional limitations inherent to their system. In this situation it is possible to decide the 
proper light-source intensity and position, imaging camera position, depth of focus, effective 
focal length, field of view, and CCD camera resolution" and page 30 "visual geometry 
inspection" and page 35 "searches the image for hot spots". 

1 17. C-communication interface... between the user interface and the hardware 
component simulation system is inherently disclosed by Stevenson at page 32 "functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution". Note that the displays at pages 29 and 30 
inherently have a communication interface between the display monitor and the simulation 
system. Similarly, the keyboard and/or mouse which is used to interact with the toolbars 
inherently has a communication interface between the display monitor and the simulation 
system. 

118. D-inputting CAD data is disclosed by Stevenson at page 30 "This allows objects 
characterized in the Rhinoceros 3-D modeling program to be imported into ASAP for further 
analysis." 

119. E- focus-dependent synthetic image... based on at least 2 of a current state of the 
user-alterable control elements [focus], the current lens system representation of the 
first portion [depth of field], and the current state representation of the second portion 
[relative position] is disclosed by Stevenson at page 32 "This type of analysis also permits 
designers to dissect the functional limitations inherent to their system. In this situation it is 
possible to decide the proper light-source intensity and position, imaging camera position, 
depth of focus, effective focal length, field of view, and CCD camera resolution". 

120. F-displaying the current focus-dependent synthetic image is disclosed by Stevenson at 
page 29 "Quality-control optical vision systems can be conveniently simulated and analyzed 
using software technology. .. simulate and analyze almost any optical vision system with 
high geometric and photometric accuracy". Note the car models displayed (synthetic 
images) at page 29 and page 30. 

121. G-operating at least one control element which is operable to perform an image 
inspection operation is disclosed by Stevenson at page 32 "This type of analysis also 
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permits designers to dissect the functional limitations inherent to their system. In this 
situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution" and page 30 "visual geometry inspection" and page 35 "searches the image for 
hot spots". 

122. Stevenson does not expressly disclose the remaining limitations (Al and A4). 

123. In claim 70, "displaying a modified current focus-dependent synthetic image... at 
least one of a) the current state of the user-alterable control elements [focus], b) the 
current lens system representation of the first portion [depth of field], and c) the 
current state representation of the second portion [relative position]" is disclosed by 
Stevenson at page 32 "This type of analysis also permits designers to dissect the functional 
limitations inherent to their system. In this situation it is possible to decide the proper light- 
source intensity and position, imaging camera position, depth of focus, effective focal length, 
field of view, and CCD camera resolution" and page 30 "visual geometry inspection" and 
page 35 "searches the image for hot spots". 

124. In claim 71, "focus-dependent synthetic image on the current state representation of 
the third portion [lighting] 99 is disclosed by Stevenson at page 32 "This type of analysis 
also permits designers to dissect the functional limitations inherent to their system. In this 
situation it is possible to decide the proper light-source intensity and position, imaging 
camera position, depth of focus, effective focal length, field of view, and CCD camera 
resolution". 

125. In claim 72, "inspection 99 is disclosed by Stevenson at page 30 "visual geometry 
inspection" and page 35 "searches the image for hot spots". 

126. MOTIVATION FOR CLAIMS 46-72. At the time the invention was made, it would 
have been obvious to a person of ordinary skill in the art to use Robotics and Thomas to 
modify Stevenson. Stevenson discloses a highly detailed machine vision simulation (with 
focus), which is intended to design, and analyze, and optimize actual physical machine vision 
systems. One of ordinary skill in the art would be motivated to implement the simulation 
results from Stevenson in a modern manufacturing environment by using Robotics' robots 
(with associated computers) to physically move the camera and lights around a car (for 
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example), much in the same way that the Robotics paint gun is moved at the end of a robot 
arm. Note that many of Stevenson's figures are car related, and that Robotics* paint guns are 
commonly used to paint new cars. In view of the figures, Stevenson's simulation appears 
especially intended for analyzing the fresh paint in new cars. 

127. Further, one of ordinary skill in the art would be motivated to use Thomas to make the 
user interface (including graphical user interface, and physical user interface, and related 
instructions) of the simulation similar to the user interface of the actual physical system being 
simulated for three reasons: first, it saves time and money to use the user interface software 
module from the actual physical system rather than to devise and design a new interface for 
the simulation. Second, it is preferable to use the same interface in simulation in order to 
more realistically train the human operator. Third, using the same interface in simulation 
facilitates future training of the actual physical system by the operator (per Robotics page 54 
where the operator is training the robot using a "teach pendant"). Thus, the simulation may 
be used to "train" both the operator and the actual physical system. 

Additional Cited Prior Art 

128. The following US patents or publications are hereby cited as prior art, but have not been 
used for rejection. Applicant should review these carefully before responding to this office 
action. US Patent 4,471,448 by Williams. Note that Willams FIG 7 element 702 
ANALYTICAL MODEL discloses a simulation of machine vision system with focus (the 
Hubble Telescope), and that said simulation results are used to control element 700 
PHYSICAL SYSTEM which is the actual Hubble Telescope. 

Conclusion 

129. All pending claims stand rejected against prior art, and under 35 USC 1 12. 

Communication 

130. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Eduardo Garcia-Otero whose telephone number is 703-305- 
0857. The examiner can normally be reached on Tuesday through Friday from 9:00 AM to 
8:00 PM. If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor, Kevin Teska, can be reached at (703) 305-9704. The fax phone number for this 
group is 703-872-9306. Any inquiry of a general nature or relating to the status of this 
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application or proceeding should be directed to the group receptionist, whose telephone 
number is (703) 305-3900. 

* * * * 





